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The  SPADE  theory  uses  linguistic  formalisms  to  model  the  program  planning 
and  debugging  processes.  The  theory  has  been  applied  to  constructing  a 
grammar-based  editor  in  which  programs  are  written  in  a structured  fashion, 
designing  an  automatic  programming  system  based  on  an  Augmented  Transition 
Network,  and  parsing  protocols  of  programing  episodes. 

The  SPADE  theory  begins  with  a taxonomy  of  basic  planning  concepts 
covering  strategies  for  identification,  decomposition  and  reformulation.  A 
handle  is  provided  for  recognizing  interactions  between  goals  and  deriving  a 
linear  solution. 

A complementary  theory  of  rational  bugs  and  associated  repair  techniques 
is  also  provided.  SPADE  introduces  a new  data  structure  to  facilitate 
debugging  --  the  derivation  tree  of  the  program.  The  SPADE  editor  makes  this 
structure  available  to  the  programmer. 

The  SPADE  theory  generalizes  recent  work  in  Artificial  Intelligence  by 
Sussman  and  Sacerdotl  on  automatic  programing,  and  extends  the  theory  of 
program  design  developed  by  the  Structured  Programing  movement.  It  provides 
a more  structured  information  processing  model  of  human  problem  solving  than 
the  production  systems  of  Newell  and  Simon,  and  articulates  the  type  of 
problem  solving  curriculum  advocated  by  Papert's  Logo  Project. 


This  report  describes  research  done  at  the  Artificial  Intelligence  Laboratory  of 
the  Massachusetts  Institute  of  Technology.  The  automatic  problem  solving  aspect  of 
this  research  was  supported  by  the  Advanced  Research  Projects  Agency  of  the  Departarant 
of  Defense  under  Office  of  Naval  Research  contract  N00014-75-C-0643,  the  educational 
aspect  by  the  National  Science  Foundation  under  grant  C40708X,  and  the  protocol 
analysis  aspect  by  the  Intelligent  Instructional  Systems  Group  at  Bolt  Beranek  and 
Newman,  under  contract  number  HDA  903-76-C-0108  jointly  sponsored  by  Advanced  Research 
Projects  Agency,  Air  Force  HDlNir  Resources  Laboretory,  Army  Research  Institute,  and 
Naval  Personnel  Research  A Development  Center. 

*This  is  an  expanded  version  of  a paper  submitted  to  the  Fifth  International  Joint 
Conference  on  Artificial  Intelligence. 
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1.  A Hultl-Faceted  Approach 

This  paper  provides  an  overview  of  three  separate  contexts  in  which  the  SPADE 
theory  is  being  simultaneously  developed  and  tested  --  computer  uses  In  education, 
automatic  programming  (a  traditional  AI  arena),  and  protocol  analysis  (the  domain  of 
information  processing  psychology).  Our  experience  has  been  that  a powerful  synergism 
results  from  this  multi-faceted  approach. 

1.  The  Education  Context:  an  editor  has  been  implemented  that  encourages  students 
to  define  and  debug  programs  in  terms  of  explicit  SPADE  design  choices.  The  editor 
provides  a structured  programming  environment  based  on  a detailed  theory  of  the 
processes  involved  in  coherently  structured  problem  solving. 

2.  The  AI  Context:  an  automatic  programmer  called  PATN  has  been  designed  using  an 
augmented  transition  network  embodiment  of  the  SPADE  theory.  This  results  in  a unified 
framework  which  clarifies  recent  work  on  planning  and  debugging  by  Sacerdoti  [1979]  and 
Sussman  [1975]. 

3.  The  Psychology  Context:  a parser  called  PAZATN  has  been  designed  that  applies 
the  SPADE  theory  to  the  analysis  of  programming  protocols.  PAZATN  produces  a parse  of 
the  protocol  that  delineates  the  planning  and  debugging  strategies  employed  by  the 
problem  solver.  PAZATN  extends  the  series  of  automatic  protocol  analyzers  developed  at 
Carnegie-Hellon  University  (Waterman  & Newell  1972,  1973;  Bhaskar  A Simon  1976]. 

Hand-simulations  of  PATN  and  PAZATN  on  elementary  programming  problems  and 
Informal  experiments  with  the  SPADE  editor  attest  to  the  theory's  cogency  in  accounting 
for  a wide  range  of  planning  and  debugging  techniques  [Goldstein  A Hiller  1976a, b; 
Hiller  A Goldstein  1976b, c,d]. 


2.  A Linguistic  Analogy 

In  developing  a represention  for  problem  solving  techniques,  we  have  been  guided 
by  an  analogy  to  computational  linguistics,  for  three  reasons. 


1.  The  concepts  and  algorithms  of  computational  linguistics,  though  originally 
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Intended  to  explain  the  nature  of  language  per  se,  supply  perspicuous  yet  powerful 
descriptions  of  complex  computations  in  general.  Problem  solving,  as  a complex 
process,  benefits  from  the  application  of  these  tools. 

2.  Computational  linguistics  decomposes  computations  into  syntactic,  semantic,  and 

pragmatic  components.  This  decomposition  clarifies  the  explanation  of  complex 
processes  when  viewed  in  the  following  manner:  syntax  formalizes  the  range  of  possible 

decisions;  semantics  the  problem  description,  and  pragmatics  the  procedural 
relationship  between  the  two. 

3.  Computational  linguistics  has  undergone  an  evolution  of  procedural  formalisms, 
beginning  with  finite  state  automata,  later  employing  recursive  transition  networks 
(context  free  grammars),  next  moving  on  to  augmented  transition  networks,  and 
culminating  in  the  current  set  of  theories  Involving  frames  [Minsky  1975, 
Winograd  1975,  Schank  1975].  Each  phase  captured  some  properties  of  language,  but  was 
incomplete  and  required  generalization  to  more  powerful  and  elaborate  formalisms. 
Following  this  evolutionary  sequence  illuminates  the  complexity  of  language  theory.  We 
have  pursued  a similar  evolutionary  approach  to  clarify  the  complexity  of  problem 
solving  processes. 


To  date,  our  theory  of  design  has  evolved  through  the  following  stages:  we 
initially  explored  context  free  grammars  for  planning  and  debugging,  and  subsequently 
their  generalization  to  ATN's;  we  examined  the  metaphor  of  protocol  analysis  as 
parsing,  initially  using  the  planning  and  debugging  grammars  to  reveal  the  constituent 
structure  of  protocols  and  later  using  the  derivations  produced  by  the  ATN  formalism; 
and,  finally,  our  most  recent  work  has  studied  the  use  of  a chart-based  parser  to 
discover  these  analyses. 


3.  A Grammatical  Theory  of  Plannlnc 


The  basis  for  SPADE  is  a taxonomy  of  frequently  observed  planning  concepts 
(fig.  1).  We  arrived  at  this  taxonomy  by  introspection,  by  examining  problem  solving 
protocols  [Hiller  A Goldstein  1976b],  by  studying  the  literature  on  problem  solving 
I Polya  1957,  1965,  1968;  Newell  ft  Simon  1972;  Sussman  1975;  Sacerdotl  1975],  and  by 
( enumerating  techniques  for  finding  procedural  solutions  to  problems  expressed  as 
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predicate  calculus  formulae  [Emden  A Kowalski  1976].  This  last  criterion  demonstrates 
that  the  taxonomy  is  currently  incomplete  --  for  example,  techniques  for  handling 
disjunctions  have  not  yet  been  analyzed  thoroughly  enough  to  warrant  inclusion. 
However,  the  taxonomy  is  adequate  for  a wide  range  of  elementary  programming  problems. 
Future  research  will  investigate  additional  planning  techniques. 


There  are  three  major  classes  of  plans  in  the  taxonomy:  identification, 

decomposition,  and  reformulation.  Identification  means  recognizing  a problem  as 
previously  solved.  Decomposition  refers  to  strategies  for  dividing  a problem  into 
simpler  sub-problems.  Reformulation  plans  alter  the  problem  description,  seeking  a 
representation  which  is  more  amenable  to  identification  or  decomposition.  The  figure 
indicates  how  these  classes  of  plans  are  further  subdivided  in  the  SPADE  theory. 


Planning,  according  to  the  theory,  is  a process  in  which  the  problem  solver 
selects  the  appropriate  plan  type,  and  then  carries  out  the  subgoals  defined  by  that 
plan  applied  to  the  current  problem.  From  this  viewpoint,  the  planning  taxonomy 
represents  a decision  tree  of  alternative  plans.  The  decision  process  can  be  modeled 
by  the  context  free  grammar  given  below.  The  grammar  explicitly  states  which  planning 
rules  Involve  recursive  application  of  solution  techniques  to  subgoals:  setup, 
interface,  mainstep,  cleanup,  and  parallel. 


(The  rules  of  the  grammar  are  written  using  the  following  syntax:  "|"  is 
disjunction,  Is  ordered  conjunction,  "ft*  is  unordered  conjunction, 
■<...>*•  is  Iteration,  [...]  is  optlonallty,  and  a lower  case  English  phrase 
enclosed  in  quotation  marks  (e.g.,  "repeat  step”)  describes  a lexical  item 
which  is  not  further  expanded  in  the  grammar.) 


IDENTIFY 

DEFINED 

DECOHPOSE 

CONJUNCTION 

SEQUENTIAL 

PARALLEL 


->  IDENTIFY  | DECOHPOSE  | REFORHULATE 
->  PRIHITIVE  | DEFINED 
->  "call  user  subprocedure*  ft  PLAN 
->  CONJUNCTION  | REPETITION 
->  SEQUENTIAL  | PARALLEL 

->  [SETUP]  ♦ <HAINSTEP  ♦ [ INTERFACE])*  ♦ [CLEANUP] 
->  <PLAN>* 


SETUP 


->  PLAN 
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HAINSTEP 

INTERFACE 

CLEANUP 

REPETITION 

ROUND 

ITER-PLAN 

TAIL-RECUR 


->  PLAN 
->  PLAN 
->  PLAN 

->  ROUND  | RECURSION 

->  ITER-PLAN  | TAIL-RECUR 

->  "repeat  step"  ♦ SEQ 

->  "stop  step"  ♦ SEQ  ♦ "recursion  step” 


The  SPADE  theory  Is  not  restricted  to  any  particular  donain.  However,  to  provide 
concrete  examples,  we  have  concentrated  on  problems  from  elementary  Logo  graphics 
programming  [Papert  1971].  This  domain  was  chosen  because  of  the  availability  of 
extensive  data  on  the  performance  of  students  writing  "turtle"  programs  to  draw  simple 
pictures.  The  grammar  rules  for  primitives  in  this  domain  are: 


PRIHITIVE 

VECTOR 

ROTATION 

PENSTATE 


->  VECTOR  | ROTATION  | PENSTATE 
->  (FORWARD  | BACK)  ♦ <number> 
->  (LEFT  | RIGHT)  ♦ <number> 

->  PENUP  | PEN DOWN 


A typical  task  undertaken  by  beginners  in  the  Logo  environment  is  to  draw  a wishingwell 
picture  using  the  computer. 


Fig.  2 — A Logo  Wishingwell 

Fig.  3 illustrates  a solution  to  the  wishingwell  problem  with  its  hierarchical 
annotation  according  to  our  planning  granaar. 


The  grammar  characterizes  the  decision  process  Involved  in  selecting  plans  from 
the  taxonomy.  We  illustrate  its  utility  in  the  next  two  sections  by  constructing  an 
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editor  that  embodies  the  grammar  and  analyzing  debugging  In  terms  of  the  grammar . Then 
we  show  how  the  grammar  can  be  augmented  to  Include  not  only  the  syntax  of  plans,  but 
their  semantics  and  pragmatics  as  well. 

4.  SPADE-0,  A Planning  Assistant 

One  reason  for  calling  our  theory  of  planning  and  debugging  structured  Is  to 
emphasize  the  link  between  our  research  and  the  Structured  Programming  movement.  Dahl, 
Dljkstra,  and  Hoare  [197Z]  call  for  a style  of  programming  which  reflects  coherently 
structured  problem  solving.  But  a detailed  formalization  of  what  this  style  entails  Is 
lacking.  Our  efforts  In  this  direction,  therefore,  supplement  the  work  of  Dljkstra  and 
others.  The  SPADE-0  editing  environment  serves  as  an  example. 

How  can  we  judge  whether  a particular  grammar  of  plans  captures,  at  some  level  of 
abstraction,  the  set  of  planning  decisions  involved  in  solving  problems  for  some 
domain?  One  methodology  we  employ  is  to  incorporate  the  grammar  Into  an  editor  ( SPADE - 
0)  whose  purpose  is  to  augment  and  direct  the  capabilities  of  a human  user.  The 
critical  question  then  becomes  determining  the  extent  to  which  such  a system  aids  or 
hinders  the  user. 

Suppose  a problem  solver  Is  defining  a Logo  program  for  drawing  the  wlshlngwell 
shown  earlier.  In  SPADE-0,  this  is  accomplished  by  applying  the  planning  gramnar  In 
generative  mode: 

la.  What  Is  the  name  of  your  top  level  procedure? 

lb.  >VW 

Za.  The  rule  is:  PLAN  ->  IDENTIFY  | DECOHPOSE  | REFORMULATE. 

What  now? 

2b.  DECOMPOSE. 


3a.  The  rule  Is:  DECOHPOSE  ->  CONJUNCTION  | REPETITION 
3b.  CONJUNCTION 


4a.  The  rule  is:  CONJUNCTION  ->  SEQUENTIAL. 


' 

■ 


► 

: 

[.  I 

I . 

i. 

IH 
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The  rule  is:  SEQUENTIAL  ->  [SETUP]  ♦ <MAINSTEP  ♦ [INTERFACE]/  ♦ [CLEANUP]. 

Do  you  wish  to  define  an  optional  SETUP? 

4b.  Later. 

\ 

* 

• • • • 


In  this  way,  SPADE-0  encourages  users  to  articulate  their  design  decisions  in  top- 
down  order.  At  the  same  time,  the  system  allows  the  user  to  escape  from  this  strict 
discipline  if  an  alternative  solution  order  seems  preferable.  This  was  illustrated  by 
the  user's  "later"  instruction.  "Later"  suspends  the  current  goal  for  later  solution. 

SPADE-0  was  implemented  by  assigning  an  interpretive  procedure  to  each  grammatical 
operator.  In  essence,  the  editor  is  a bookkeeper  for  the  user’s  goal  tree.  Despite 
its  simplicity,  we  see  three  useful  applications  of  the  program. 


1.  From  an  educational  standpoint,  the  editor  encourages  students  to  articulate 
their  problem  solving  strategies.  The  fundamental  hypothesis  of  the  Logo  Project,  as 
presented  by  Papert  i-n  Teaching  Children  Thinking  [1971],  is  that  such  articulate 
problem  solving  is  beneficial  to  the  learner.  Tfu  SPADE  editor,  with  its  extreme  form 
of  articulation,  provides  an  experimental  vehicle  for  evaluating  Papert 's  claim:  do 
students  exposed  to  SPADE  progress  faster  in  learning  Logo  than  controls  whose  problem 
solving  is  more  tacit? 


t i 

:•{ 

I 

¥ 


2.  From  an  AI  standpoint,  its  use  will  indicate  whether  the  planning  graimaar  is 
adequate,  or  whether  certain  plans  are  not  present  that  competent  problem  solvers  feel 
are  necessary. 

3.  From  a psychological  standpoint,  we  will  collect  transcripts  of  individuals 
using  the  editor  and  formulate  personal  granmars  based  on  the  particular  rules  usually 
employed  by  each  user.  The  personal  grammar  will  model  the  problem  solving  skills  of 
that  individual.  In  the  past  we  have  manually  analyzed  protocols  from  standard  Logo. 
SPADE  protocols,  with  their  explicit  planning  choices,  can  be  far  more  revealing. 
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5.  RAID,  A Debugging  Assistant 

SPADE  Includes  a theory  of  debugging.  Such  a theory  is  essential,  since  problem 
solvers  must  often  formulate  plans  in  the  face  of  imperfect  knowledge  and  limited 
resources.  Under  such  circumstances,  even  carefully  reasoned  Judgments  may  be 
mistaken. 

Given  a grammatical  theory  of  planning,  debugging  can  be  analyzed  as  the 

localization  and  repair  of  errors  in  applying  grammar  rules  during  planning.  Since  our 

* 

planning  rules  were  constructed  from  operators  for  conjunction,  disjunction,  and 
optionallty,  three  basic  classes  of  errors  arise: 

1.  syntactic  bugs,  in  which  the  basic  grammar  is  violated,  such  as  when  a 
required  conjunct  is  missing; 

2.  semantic  bugs,  in  which  a semantic  constraint  arising  from  the  particular 
problem  is  violated,  such  as  when  a syntactically  optional  constituent,  needed 
because  of  the  semantics  of  the  particular  problem,  is  missing; 

3.  pragmatic  bugs,  in  whicH  an  inappropriate  selection  from  a set  of  mutually 
exclusive  disjuncts  is  made. 

Fig.  4 provides  examples  of  these  bug  types.  Although  additional  categories 
(including  execution  and  efficiency  bugs)  must  be  defined  to  make  this  taxonomy  of  bugs 
complete,  these  classes  are  adequate  to  characterize  many  examples  which  arise  in 
elementary  programming. 

RAID  is  a debugging  assistant  for  SPADE-0  implemented  on  the  basis  of  this 
taxonomy  of  errors.  As  SPADE-0  prompts  the  user  through  planning  process,  those 
choices  which  might  lead  to  syntactic,  semantic,  or  pragmatic  bugs  are  recorded  by  the 
system.  In  subsequent  debugging  episodes,  RAID  alerts  the  user  to  the  relevant  bug 
warnings,  thereby  aiding  in  diagnosis.  Horeover,  the  SPADE-0  planning  editor- can  then 
be  relnvoked  to  guide  repair  of  the  bug  by  resuming  planning  in  the  context  of  the 
culpable  decision.  An  interaction  with  RAID  that  would  arise  from  premature  execution 
of  a program  is  illustrated  below: 


; » 


o 


)..  u....  ....1.1  . 
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FIGURE  4 - SYNTACTICALLY  INCORRECT  I'l.AN 

A NECESSARY  CONJUNCT  IS  MTSSINC. 


TO  WW 

10  TRIANGLE USE 


END 


TD-PLAN 


WW 


GET 


???  TRIANGLE  UNDEFINED  ??? 


("GET"  MISSING.  UNGRAMMATICAL  PLAN. 
DEBUG  BY  COMPLETING  PLAN.) 


GET  TRIANGLE  FILE 


FIGURE  4 - SEMANTICALLY  INCORRECT  PLAN 
AN  OPTIONAL  CONJUNCT  IS  MISSING 


FOR  EXAMPLE:  "WW"  MISSING  INITIAL  SETUP,  AND  INTERFACE  FOR  PCM. 


TO  WW 

10  WELL MAINSTEP-i 

20  POLE  — MAINSTEP 


lSEQ-PLAN 


FIGURE  4 - PRAGMATICALLY  INCORRECT  PLAN 
AN  INCORRECT  DISJUNCT  HAS  BEEN  SELECTED 


TO  SQUARE-INSIDE-TRIANGLE 
10  SQUARE 
20  TRIANGLE 
END 


LINEAR  FLAN  -- 
SQUARE  AND  TRIANGLE- 
DESIGNED 
INDEPENDENTLY . 


INTENDED  PICTURE: 


ACTUAL  PICTURE 
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The  user  has  continued  defining  the  wishingwell  program  begun  earlier,  with 
its  current  definition  as  shown  in  fig.  4b.  Although  an  initial  SETUP  end  ew 
I UT ERF ACE  have  not  get  been  supplied,  the  user  requests  execution  of  the 


la.  ...  What  now? 

lb.  >RUN  WISHINGWELL. 

2a.  OR.  ( Fig  4b  shows  the  resulting  picture .) 

What  now? 

2b.  >Diagnose 

3a.  The  rule  is:  DIAGNOSES  <PARSE  | CODE  | MODEL  | PROCESS)* . 

What  now? 

3b.  >Parse 

4a.  The  rule  is:  PARSE  ->  SYNTAX  | SEMANTICS  | PRAGMATICS. 

What  now? 

4a.  )Semantics 


Sa.  The  following  optional  constituents  are  nissing  in  WW: 

The  SETUP  and  the  INTERFACE  between  WELL  and  POLE. 

What  now? 

5b.  >Repair 

j 

6a.  The  rule  is:  REPAIR  ->  COMPLETE  | REPLACE. 

6b.  >Complete  Interface  between  well  and  pole. 


7a.  The  rule  is:  PLAN  ->  IDENTIFY  | DECOMPOSE  | REFORMULATE 

7b.  > ... 

In  this  example,  the  interaction  with  SPADE  was  again  guided  by  a grammar;  this 
debugging  grammar  is  given  below. 
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DIAGNOSE 


PROCESS 


■>  < PARSE  | CODE  | MODEL  | PR0CES5>* 
■>  <ASK  | TRACE)* 

■>  "print  definition" 


HODEL 


->  RUN 


PARSE 


SYNTAX 


SEMANTICS 


PRAGMATICS 


REPAIR 


■>  SYNTAX  | SEMANTICS  | PRAGMATICS 
•>  "review  conjunctive  choices  in  plan" 
-)  "review  optional  choices  in  plan" 

•>  "review  disjunctive  choices  in  plan" 
■>  COMPLETE  | CORRECT 


COMPLETE 


•>  "solve  for  missing  conjunct" 


CORRECT 


-)  "choose  alternative  disjunct" 


■>  "run  code  line"  ♦ ["error  msg"]  ♦ [DEBUG] 
■>  "print  definition*  I "print  value" 


TRACE 


■>  [SELF-DOC*]  ♦ RUN* 


SELF-DOC 


->  "add  break"  | "add  print"  | "add  trace" 


This  grammar  represents  the  traditional  debugging  strategies  employed  by  programmers 
supplemented  by  the  PARSE-based  strategies  allowed  by  the  existence  of  the  SPADE 


derivation  tree. 


For  more  complex  programs,  there  will  be  too  many  potential  bug  locations  for 
RAID's  undirected  advice  to  be  helpful.  Hence,  following  SPADE-O's  structured 
bookkeeping  philosophy,  we  allow  the  user  to  set  caveats  during  planning  regarding 
those  decisions  which  appear  to  be  potential  trouble  spots.  In  this  fashion,  SPADE 
permits  an  exploratory  style  of  problem  solving,  as  illustrated  below. 


Suppose  the  mishinguell  program  is  part  of  a large  project.  Then  the  number 
of  optional  constituents  that  have  not  been  defined  mill  be  large.  The  user 
will  probably  have  some  insight  into  which  of  these  is  actually  needed.  To 
facilitate  debugging , the  user  is  encouraged  to  place  caveats  at  those 
decisions  he  suspects  may  require  later  debugging. 


6a.  Do  you  wish  to  Include  the  optional  initial  SETUP  in  WW? 


6b.  >No. 
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7a.  Do  you  wish  to  record  a caveat  here? 
7b.  >Yes. 


✓ 


;.l 


8a.  OK.  Your  pending  goals  are  ROOF,  POLE  and  WELL.  What  now? 

8b . > . . . 

Another  nechanisa  to  make  SPADE'S  and  RAID's  advice  aore  directed  Is  to  supply  the 
semantics  for  problems.  This  is  the  topic  of  the  next  section. 


6.  PATN  --  An  Augmented  Transition  Network  for  Plannim 


While  context  free  grammars  can  represent  a useful  abstraction  of  planning 
decisions,  they  have  limitations  which  prevent  them  from  providing  a complete  theory  of 
design.  They  provide  no  representation  for  the  semantics  of  the  problem  nor  for  the 
pragmatics  involved  in  choosing  one  plan  over  another.  For  this  reason,  we  have 
designed  and  are  currently  implementing  PATN,  an  augmented  transition  network  (ATN) 
problem  solver.  We  have  adopted  the  ATN  formalism  for  the  same  reasons  that  led  Woods 
[1970]  to  introduce  it  into  computational  linguistics:  the  semantic  and  pragmatic 
limitations  of  context  free  grammars. 


Fig.  5 provides  a global  view  of  PATN  [Goldstein  & Hiller  1976b].  The  topology  of 
the  network  embodies  the  planning  grammar.  Registers  contain  descriptions  of  the 
problem,  the  solution,  and  various  temporary  constructs  built  during  planning.  Arc 
predicates  supply  pragmatic  guidance  by  examining  the  registers  and  appropriately 
directing  the  planning  process.  For  example,  an  identification  plan  cannot  proceed  If 
the  problem  description  cannot  be  round  in  the  answer  library.  PATN  has  been 
successfully  hand-simulated  on  elementary  Logo  and  Blocks  World  problems. 

PATN  allows  us  to  elaborate  our  notion  of  a completed  plan  by  defining  an 
annotated  derivation  tree.  Associated  with  each  node  of  the  plan  derivation  Is  a 
snapshot  of  the  values  of  the  ATN  registers  at  the  point  in  the  planning  process  when 
that  node  was  created.  A derivation  tree  reveals  the  constituent  structure  of  the 
plan;  these  semantic  variables  reveal  the  semantic  intent.  A set  of  assertions  at 
each  node,  derived  as  instances  of  PATN's  arc  predicates,  reveal  the  pragmatic  reasons 
Tor  preferring  a given  plan  over  its  competitors.  Fig.  6 shows  the  annotated  ) 
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FIGURE  6 PATN’S  Annotated  DERIVATION  TREE  FOR  THE  WISHIWGWELL  TASK 
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derivation  tree  for  the  wlshingwell  program  given  earlier. 

An  intended  application  of  PATN  Is  as  a module  of  the  SPADE  editor  for  providing 
the  user  with  additional  direction  and  advice.  The  following  hypothetical  dialogue 
with  SPADE-1  (the  SPADE  editor  augmented  by  PATN)  illustrates  this. 

la.  Tell  me  the  name  of  your  toplevel  procedure? 

lb.  >WW 

2a.  Would  you  like  to  define  a model? 

2b.  >WISHINGWELL 

3a.  Ok.  WISHINGWELL  model  found  in  Model  Library.  The  rule  is: 

PLAN  ->  IDENTIFY  | DECOMPOSE  (REFORMULATE.  What  now? 

3b.  >ROOF  POLE  WELL 

4a.  These  subgoals  correspond  to  model  parts. 

Are  you  pursuing  a LINEAR  DECOMPOSITION  plan? 

4b.  >YES 

Sa.  The  pending  subgoals  are:  ROOF.  POLE,  WELL  and  the  preparatory  steps. 

What  new? 

5b.  >ROOF 

6a.  In  the  WW  model,  ROOF  is  described  as  a triangle. 

There  is  a TRIANGLE  procedure  in  the  answer  library. 

Would  you  like  to  use  it? 

6b.  >YES 

7a.  Ok.  The  ROOF  is  complete.  Your  pending  goals  are: 

POLE,  WELL,  and  the  preparatory  steps.  What  now? 

7b.  > ... 

PATN's  Linearization  Loop  provides  a theoretical  superstructure  for  handling 
dependent,  interacting  subgoals.  The  approach  is  to  view  a non-linearity  as  a property 
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of  the  problem  description.  That  Is,  even  though  one  description  of  the  problem  leads 
to  an  unsuccessful  refinement  process  because  of  interactions  between  subgoals,  another 
description  can  succeed  by  mating  the  dependent  constraints  explicit.  For  example, 
consider  the  SQUARE- INSIDE-TRIANGLE  of  fig.  4C.  If  the  problem  description  is: 


SQUARE  X; 
TRIANGLE  Y; 
INSIDE  X Y; 


then  a decomposition  that  draws  the  square  and  triangle  independently  and  then  attempts 
to  fit  them  together  to  achieve  the  inside  relation  will  fall.  However,  a problem 
description  of  the  following  form  allows  a successful  decomposition: 


SQUARE  X,  WITH  SIDE  * 100; 
TRIANGLE  Y,  WITH  SIDE  = 300; 
CENTER  OF  X = CENTER  OF  Y. 


The  INTERACTIONS  predicate  is  a conjunction  of  tests  on  the  model  register.  Each 
test  is  responsible  for  detecting  a given  non-linearity.  A corresponding  action 
modifies  the  model,  adding  new  statements  to  make  the  interaction  explicit.  The 
REFINEMENT  loop  is  the  repository  for  what  Sussman  [1975]  calls  the  Critics  Gallery. 
The  theoretical  progress  of  PATN  is  to  integrate  the  Critics  Gallery  concept  into  a 
theory  of  planning.  In  Sussman's  HACKER,  the  critics  gallery  and  library  of 
programming  techniques  were  separate  modules:  there  was  no  integrated  theory. 

Of  course,  at  any  point  in  time  the  system  may  be  unaware  of  a given  type  of  non- 
linearity. In  such  cases,  the  absence  of  an  interaction  test  will  lead  to  a sequential 
decomposition  that  ultimately  falls.  The  design  of  a program  for  debugging  such 
failures  is  the  subject  of  the  next  section. 

7.  DAPR  --  An  Augmented  Transition  Network  for  Debuggln 


PATN  can  make  mistakes.  That  is,  PATN  will  sometimes  introduce  what  we  term 
rational  bugs  into  its  plans,  due  to  making  arc  transitions  with  imperfect  knowledge  of 
subtleties  or  interactions  in  the  task  domain.  Hence,  PATN  must  be  equipped  with  a 
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complementary  debugging  module,  DAPR  (fig.  7). 

While  RAID  was  designed  to  assist  human  programmers  in  finding  a variety  of  bugs 
(primarily  by  plan  diagnosis),  DAPR  was  designed  to  analyze  PATN's  bugs.  DAPR  employs 
three  diagnostic  techniques:  model,  process,  and  plan  diagnosis.  Model  diagnosis  is 
the  basic  technique.  It  amounts  to  comparing  the  effects  of  executing  a plan  to  a 
formal  description  of  its  goals,  to  determine  if,  and  in  what  fashion,  the  plan  has 
failed.  Another  DAPR  technique,  based  on  Sussman’s  HACKER  [1975],  is  examining  the 
state  of  the  process  at  the  time  of  the  error  manifestation.  Plan  diagnosis  is  a DAPR 
first.  It  is  accomplished  by  examining  the  caveats  variable  associated  with  various 
nodes  of  PATN's  annotated  plan  derivation. 

DAPR  will  also  be  used  to  provide  additional  guidance  to  RAID.  This  illustrates 
the  synergism  possible  when  educational,  psychological  and  AI  facets  of  a cognitive 
theory  are  studied  in  an  integrated  fashion.  This  integration  is  further  exemplified 
In  the  next  section  when  we  apply  the  SPADE  theory  to  protocol  analysis. 


8.  PAZATN.  an  Automatic  Protocol  Analyzer 


As  soon  as  one  has  an  heuristically  adequate  theory  of  design,  it  is  natural  to 
ask,  "Can  the  theory  provide  an  account  of  how  people  solve  problems?*.  An 
experimental  technique  we  employ  for  answering  this  question  is  the  analysis  of 
protocols  collected  during  problem  solving  sessions.  By  adopting  this  methodology  we 
follow  the  precedent  established  in  seminal  protocol  analysis  studies  conducted  at 
Carnegie  Hellon  University  [Newell  ft  Simon  1972;  Waterman  ft  Newell  1972,  1973; 

Bhaskar  ft  Simon  1976].  Our  work  extends  their  approach  along  three  dimensions. 

1.  With  the  exception  of  the  recent  Bhaskar  ft  Simon  effort,  the  CHU  studies  have 
been  restricted  to  very  limited  domains  such  as  cryptarlthmetic.  Rather  than  limiting 
the  task  domain,  we  limit  the  range  of  responses.  Typically  protocols  are 
transcriptions  of  think-aloud  verbalizations;  we  focus  on  the  more  restricted 
interactions  arising  from  a problem  solving  session  at  a computer  console.  The 
analysis  task  in  this  setting  is  to  interpret  user  actions  --  editing,  executing, 
tracing,  etc.  — in  terms  of  the  SPADE  theory  of  planning  and  debugging. 


FIGURE  7 DAPR:  PATH'S  DEBUGGING  ATH 
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Z.  The  CHU  theory  centers  on  the  production  systems  model.  Although  productions 
are  Turing  universal,  they  tend  to  result  In  a less  structured  program  organization 
than  the  linguistic  formalisms  of  the  SPADE  theory.  In  PATN,  each  arc  transition, 
consisting  of  a predicate  and  an  action,  can  be  thought  of  as  a production.  However, 
PATN  organizes  these  productions  into  local  contexts,  each  of  which  consists  of  the 
arcs  exiting  from  a given  node.  Not  all  of  the  arc  productions  are  present  at  any 
moment  in  time;  an  arc  is  present  only  when  the  problem  solver  is  at  the  relevant 
node.  In  the  production  systems  discussed  in  Human  Problem  Solving  [Newell  & 
Simon  1972],  all  of  the  productions  are  always  present  and  are  tested  in  serial  order. 

3.  CHU  analyses  are  based  on  the  problem  behavior  graph.  Pursuing  an  analogy  to 
computational  linguistics,  we  define  an  interpretation  of  a protocol  to  be  a parse  tree 
supplemented  by  semantic  and  pragmatic  annotation.  The  parse  tree  characterizes  the 
constituent  structure  of  the  protocol.  Semantic  and  pragmatic  annotation  — variables 
and  assertions  attached  to  nodes  of  the  parse  tree  --  formalize  the  problem  description 
and  the  rationale  for  particular  planning  choices.  Annotated  parse  trees  closely 
reflect  the  local  structure  of  PATN's  linguistic  problem  solving  machinery,  leading 
more  directly  to  inferences  regarding  individual  differences  than  is  evident  from 
problem  behavior  graphs. 


Ruven  Brooks  [1975]  applied  the  CHU  approach  to  the  programming  domain, 
developing  a model  of  coding  --  the  translation  of  high  level  plans  into  the  statements 
of  a particular  programming  language  --  and  testing  the  model  by  analyzing  protocols. 
His  model  is  a set  of  production  rules  whose  conditions  match  the  patterns  of  plan 
elements  and  whose  actions  generate  code  statements.  Protocols  are  analyzed  manually, 
with  the  experimenter  attempting  to  infer  the  plan  which  is  then  expanded  by  the 
production  system  into  code  paralleling  that  of  the  protocol.  The  processes  of 
understanding  the  problem,  generating  the  plan,  and  debugging  are  not  formalized. 
SPADE  goes  beyond  this,  in  that  it  can  be  used  to  parse  protocols  and  that  the  parse 
constitutes  a formal  hypothesis  regarding  not  only  the  coding  knowledge  but  also  the 
planning  and  debugging  strategies  employed  by  the  problem  solver. 


[Hiller  ft  Goldstein  1976b]  provides  a detailed  example  of  such  analysis  being 
performed  by  hand.  The  example  is  a segment  from  a protocol  several  hundred  lines  long 
in  which  a high  school  student  uses  the  Logo  turtle  to  draw  the  letters  of  his  name. 
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By  examining  the  grammar  rules  present  in  the  derivation,  we  can  readily  observe 
various  properties  of  the  student's  problem  solving  such  as:  reliance  on  certain 
planning  choices  to  the  exclusion  of  others  (e.g.,  the  student  employed  Iteration,  but 
never  recursion);  the  misuse  of  certain  optional  constituents  (e.g.,  a setup  was 
usually  included  in  each  procedure  even  when  it  was  unnecessary);  and  certain 
situations  where  his  problem  solving  violates  the  graamar  and  hence  is  susceptible  to 
syntactic  errors  (e.g.,  programs  were  often  executed  before  their  subprocedures  had 
been  defined). 

Just  as  a context  free  grammar  is  incomplete  as  a theory  of  planning,  likewise  a 
parse  is  only  a partial  analysis  of  a protocol.  The  theory  of  annotation  developed  in 
the  PATN  work  led  us  from  describing  only  the  syntactic  structure  to  more  complete 
analyses  of  protocols:  an  interpretation  of  a protocol  is  the  selection  of  a 
particular  annotated  PATN  plan  derivation.  Fig.  8 shows  such  an  analysis  of  a 
simplified  protocol  in  which  a wishlngwell  program  is  defined,  executed  and  debugged. 


PAZATN  is  a chart-based  parser  [Kay  1973;  Kaplan  1973]  being  Implemented  to 
Interpret  protocols  in  terms  of  PATN's  annotated  plan  derivations  [Biller  ft 
Goldstein  1976d].  It  will  operate  by  causing  PATN  to  deviate  from  its  preferred 
approach  in  response  to  bottom-up  evidence  (fig.  9).  By  taking  advantage  of  powerful 
parsing  strategies  developed  in  research  on  speech  understanding  [Lesser  et  al.  197S; 
Paxton  ft  Robinson  1975],  as  well  as  the  economical  chart  representation  of  ambiguities, 
PAZATN  has  been  successfully  hand-simulated  on  approximately  10  Logo  protocols. 
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PAZATN  will  operate  by  matching  PATN-generated  plans  with  protocol  data.  Two 
charts  will  be  used  to  represent  alternative  interpretations.  The  PLANCHART  keeps 
track  of  the  set  of  plausible  subgoals  which  have  been  proposed  by  PATN.  Fig.  10  shows 
a planchart  for  a wishlngwell  in  which  PATN  has  proposed  two  alternative 
decompositions.  The  structure  is  a chart  because  it  shares  substructures,  as 
exemplified  by  the  common  solution  to  the  WELL  subgoal  pointed  to  by  both  wishlngwell 
decompositions.  The  DATACHART  records  the  state  of  partially  completed 
Interpretations.  Fig.  11  shows  how  the  datachart  links  events  into  the  planchart  for  a 
PAZATN  interpretation  of  the  wishlngwell  protocol  given  earlier. 


These  charts  are  grown  as  follows.  First  PAZATN  requests  PATN  to  generate  its 
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FIGURE  9 


TOP  LEVEL  ORGANIZATION  OF  THE  PROTOCOL  ANALYZER 
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FIGURE  10  PLANCHART  OF  ALTERNATIVES  FOR  WW 
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FIGURE  11  DATACHART  LINKING  PROTOCOL  EVENTS  TO  PLANCHART  LEAVES 
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most  plausible  plan.  This  plan  is  inserted  into  the  PLANCHART.  Then  protocol  events 
are  analyzed  one  by  one,  and  matched  with  subgoals  in  the  PATN  plan.  The  match  is 
recorded  in  the  DATACHART.  If  no  plausible  matches  are  found,  PATN  is  asked  to 
generate  the  next  most  plausible  plan.  The  PLANCHART  is  thereby  extended.  Common 
subgoals  share  the  same  structure  in  the  chart. 

At  first,  PAZATN  will  be  Implemented  Interactively,  with  the  user  --  a 
psychologist  analyzing  a protocol  ~ directing  PAZATN  to  select  different  PATN  plans. 
This  follows  the  Incremental  implementation  strategy  used  in  two  of  the  CHU  protocol 
analyzers  [Waterman  ft  Newell  1972;  Bhaskar  ft  Simon  1976].  PAZATN,  even  in  its  early 
interactive  stages,  should  provide  strong  evidence  regarding  PATN's  adequacy  as  a 
cognitive  theory. 

PAZATN  will  also  be  tested  in  the  SPADE  context.  Below  is  a hypothetical  dialogue 
with  SPADE-2,  representing  the  original  SPADE-0  augmented  by  both  PATN  and  PAZATN. 

la.  We  are  now  solving  for  a WISHINGWELL.  Your  pending  subgoals  are: 

ROOF,  POLE,  WELL,  and  the  interfaces.  What  now? 

lb.  > SQUARE 

2a.  OR.  I am  assuming  that  WELL  has  been  solved  by  identifying  it 
with  the  previously  solved  SQUARE  procedure.  What  now? 

2b.  > ... 


PAZATN  will  Increase  the  editor's  flexibility  in  handling  ambiguous  events,  and  In 
alleviating  what  might  seem  to  some  users  to  be  an  excessive  allocation  of  time  and 
effort  to  the  planning  phase.  Ultimately  we  expect  SPADE-2  to  provide  a programming 
environment  of  value  to  professional  as  well  as  student  programmers. 

9.  Conclusions 


An  outline  of  the  SPADE  theory  of  design  has  been  presented.  One  theme  has  been 
the  synergism  achievable  in  a multi-faceted  research  effort.  Programs  have  been 
designed  and  are  being  implemented  for  testing  SPADE  in  educational,  psychological  and 
I ; AI  contexts.  The  AI  dimension  involves  the  development  of  a spectrum  of  possible 
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theories  of  cognition.  The  psychological . and  educational  distensions  supply  feedback 
regarding  which  parts  of  a theory  are  rejected  by  competent  human  problem  solvers. 
Furthermore,  the  psychological  dimension  would  be  incomplete  were  it  not  to  address  the 
issue  of  learning,  while  the  educational  dimension  characterizes  the  trajectory  of  the 
student's  cognitive  state  through  time. 

A second  theme  has  been  the  exploitation  of  concepts  and  algorithms  from 
computational  linguistics:  graanars,  ATN's,  derivation  trees,  search  strategies  from 

speech  understanding,  chart-based  parsers.  Computational  linguistics  is  also 
responsible  for  suggesting  the  propitious  decomposition  of  problem  solving  processes 
into  components  involving  syntactic,  semantic  and  pragmatic  knowledge. 

We  believe  that  our  unified  approach  to  AI,  psychology  and  education  represents  a 
new  research  paradigm  offering  the  potential  for  considerable  progress  in  all  three 
fields.  But  much  remains  to  be  done.  Although  all  of  the  programs  have  been  designed 
and  hand- simulated,  only  the  SPADE-0  editor  has  been  implemented.  Furthermore,  while 
SPADE  has  been  applied  to  Logo  graphics,  blocks  world  and  elementary  calculus  problems, 
it  has  not  yet  been  exercised  in  enough  contexts  to  prove  its  generality.  If  this 
research  project  succeeds,  a new  clarity  will  have  been  brought  to  the  study  of  problem 
solving.  Even  if  it  fails,  the  reasons  for  the  failures  should  provide  useful 
insights.  In  any  case,  it  has  already  unified  the  treatment  of  plans  and  bugs,  a 
significant  stride  for  a theory  of  problem  solving. 
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